Neural network based predication and optimization for groundwater / surface water system

ABSTRACT

The present invention relates to a method and apparatus, based on the use of a neural network (NN), for (a) predicting important groundwater/surface water output/state variables, (b) optimizing groundwater/surface water control variables, and/or (c) sensitivity analysis, to identify physical relationships between input and output/state variables used to model the groundwater/surface water system or to analyze the performance parameters of the neural network.

RELATED APPLICATION/CLAIM OF PRIORITY

[0001] This application is related to and claims priority fromProvisional Application Serial No. 60/347,626, filed Oct. 22, 2001.

INTRODUCTION

[0002] The present invention relates to a method and apparatus, based onthe use of a neural network (NN), for (a) predicting importantgroundwater/surface water output/state variables, (b) optimizinggroundwater/surface water control variables, and/or (c) sensitivityanalysis, to identify physical relationships between input and outputvariables used to model the groundwater/surface water system or toanalyze the performance parameters of the neural network.

BACKGROUND

[0003] Management and protection of water resources around the world isbecoming critically important. Overuse and contamination combined withincreasing water demand are diminishing the quantity and quality ofwater available for human use. It has been estimated that by year 2025,over 35% of the world population will face chronic water shortageproblems. As a result, it is imperative that water resources aredeveloped and managed in a sustainable and equitable manner.

[0004] Adverse environmental impacts like aquifer overdraft, saltwaterintrusion, wetlands dewatering, stream flow depletion, and spreading ofgroundwater contamination are often consequences of improperly managedsystems. Accurate prediction of the system states under variable control(e.g. pumping and injection rates) and random (e.g. precipitation andtemperature) variables can help ameliorate many of these environmentalproblems. In addition, optimization can be applied to utilize theresource in the most cost-effective manner while ensuringsustainability.

PREDICTION

[0005] Computer models based on physical principles are traditionallyused for simulation prediction and management of water resources. Thepredictions enable scientists and decision-makers to assess the impactsof various scenarios from which appropriate management policies can bedeveloped and implemented. The effectiveness of the policies largelydepends upon the accuracy of the model predictions. Typically, themodeler will attempt to accurately predict changes in the statevariables of the resource as a result of the imposition of human controlvariables like pumping, and random variables like climate. For example,the computer model may be used to assess the potential impact ofmunicipal well pumping on water levels near wetlands and flows instreams during drought conditions. The nature and extent of the pumpingand climate impacts will depend upon the physical attributes of thegroundwater/surface water system. For a physical-based computer model toaccurately predict changes of the state variables in response to controland random variables, the equations must adequately represent theconditions and processes of the physical system under study.

[0006] Following development of physical-based models, they are oftenlinked with or embedded into optimization programs to identify the“optimal” control policy. For the example water management exampleabove, this might be maximizing pumping of the supply wells withoutdewatering wetlands or depleting stream flows.

[0007] The most advanced physical-based models for predicting statevariables are expressed by numerical equations. Numericalgroundwater/surface water models use the laws of conservation of massand momentum to solve for the distribution of water levels or headacross the study area of interest. Once the energy field is computednumerically, the groundwater/surface water model can make otherpredictions such as hydraulic gradients, groundwater flow velocities(direction and magnitude), fluxes into or out of surface water bodies,and the fate and transport of contaminants, etc. Because numericalmodels embody the physics of flow within the numerical equations, thephysical properties of the system, as well as the boundary and initialconditions, must be adequately represented by the model to achieve anacceptable degree of accuracy.

[0008] Groundwater/surface water properties are by nature extremelyheterogeneous, exhibiting tremendous spatial variability (e.g. hydraulicconductivity values in nature range over 14 orders of magnitude) invalues over very short distances. Because of the expenses associatedwith quantifying these properties, typically a very limited number offield and/or laboratory measurements are made. Except under relativelyideal hydrogeologic conditions, assigning a “representative”distribution of these parameters across a model grid can be a difficultif not impossible task. As a result, small-scale features (e.g. lensesor fractures) that can have a significant effect on local or evenregional flow regimes are rarely known, much less incorporated into thenumerical model. Even stochastic approaches, which attempt to model thenatural heterogeneity of the subsurface conditions, have limitedapplication to real-world conditions because of simplifying mathematicalassumptions and relatively sparse field information.

[0009] Because of the inherent difficulty of determining a priori a setof representative model parameter values across the grid, calibrationhas become a standard component of model development. This consists ofsystematically adjusting model parameter values to reduce the errorbetween the computer simulated and field-measured water levels. Toeliminate the tedious trial and error calibration process, inversemodeling has been used.

[0010] Discretization of the numerical model domain into a finite numberof cells or elements limits the degree of surface/subsurfaceheterogeneity that can be captured by any model. Within each cell, asingle value for each model parameter must be assigned. In reality,real-world property values (e.g. hydraulic conductivity, river bedleakance, etc.) as represented by the model parameters change over themicroscale.

[0011] In addition, because of layering and structural features thehydraulic conductivity field in most geologic systems is inherentlyanisotropic (directionally dependent). In many modeling efforts isotropyis often assumed, despite the fact that anisotropy can exert significantcontrol on the flow field. In other cases a simple heuristic is assumedand adopted during model calibration without field or laboratory testvalidation.

[0012] Initial conditions refer to the head distribution across themodel domain at the beginning of a simulation. We select initialconditions as the final steady state head field for the calibrated modelunder average hydrologic conditions. In other cases, the final headfield for a transient (time dependent) simulation may be used.Regardless, there will always be an inherent amount of error betweeninitial conditions assigned to the model and the real-world initialconditions, serving as an additional source of prediction error.

[0013] Boundary conditions are physical boundaries such as impermeablerock or large bodies of surface water. They can be extremely difficultto quantify (e.g. mountain front recharge) and as in the real world theylargely determine the pattern of flow in the model. As with initialconditions, discrepancies between assigned and real-world boundaryconditions contribute to prediction error, particularly during transientsimulations.

[0014] A more fundamental modeling problem may arise if the physicalassumptions of the numerical model as represented by its equations donot match the natural system. Conventional groundwater flow models suchas MODFLOW (developed by the United States Geological Survey) areconstructed under conditions that do not hold necessarily in practicalcases (e.g. non-Darcian flow, fractured rock and karstic environments,etc.). The effect of underlying simplifying assumptions cannot beforeseen. Hence model accuracy is questionable. However, the simplifyingassumptions make the problem solvable, which is why they continue to beapplied in practice. Consequently, numerical groundwater flow models aresusceptible to producing relatively large predictive errors of waterlevels and other variables. This in turns affects the simulationaccuracy of predictions made using traditional groundwater/surface watermodels, and can compromise sound resource management decisions.

[0015] The present invention, which is based on a neural network (NN)approach, overcomes problems associated with traditional physical-basedmodels by accurately predicting groundwater/surface water behavior ofinterest independent of knowledge regarding expensive and difficult toquantify physical parameters, such as hydraulic conductivity, riverbedthickness, riverbed slope, and storativity. In addition, modelingproblems associated with adequate characterization of both boundary andinitial conditions and anisotropy are avoided.

[0016] The NN approach uses relatively easily measurable variables suchas water levels, pumping rates, precipitation, and temperature to learnthe behavior of interest for the groundwater/surface water system. Inaddition, the NN equations serve as a highly accurate physical-systemsimulator, and can be coupled with or embedded into an optimizationprogram for the purpose of identifying the optimal management solution.Because of its flexibility and computational speed, the NN can be usedto make predictions and manage the resource in real-time. This can beautomated by interfacing or coupling the NN/optimizer with a real-time,on-line automated data collection system (e.g. Supervisory Control andData Acquisition System, SCADA) or other type of remote data collectionand control network.

OPTIMIZATION

[0017] There are problems associated with optimization using numericalmodels, the difficulties of which are discussed below. The presentinvention overcomes these limitations. Typical groundwater/surface waterhydraulic management optimization models can be classified into theembedding method and the response matrix approach.

[0018] In the embedding method, the numerical approximations of thegoverning groundwater flow equations are embedded into a linear programas part of the constraint set. For the confined aquifer case, thegoverning equations are linear and complications are therefore avoided.However, for the unconfined case, the equations are treated as linearwith respect to the square of the hydraulic head. This approach has beenapplied to both steady (single management period) and transient(multiple time-period) cases. A major disadvantage of this method isthat the size of the constraint matrix can become extremely large.

[0019] Alternatively, the response matrix approach is the most commontechnique because of its computational efficiency. With this technique,the unit response to a unit pulse of stress is numerically computed forall possible stresses at all points of interest in the aquifer. Theseunit responses, which in groundwater problems typically represent thederivative of head with respect to pumping, are called responsecoefficients. The response coefficients are collected into a responsematrix, which is used explicitly to formulate the optimization problem.

[0020] Particular care, however, must be exercised when applying theresponse matrix approach to unconfined aquifers. Unlike the confinedaquifer case, where system linearity ensures response coefficient valuesindependent of the perturbation value (e.g. pumping increment), responsecoefficients for the unconfined (water table) aquifer are sensitive tothe perturbation value. Because of system nonlinearity, perturbation forthe unconfined case requires small increments in pumping to achievereasonable accuracy. However, as it is well known from numericalanalysis small step size in numerical differentiation results in highround off error.

[0021] A powerful advantage for performing optimization with the NNapproach is that its derived equations, whether linear or non-linear inform, can be easily embedded into or coupled to an optimization solverfor management purposes. Because the NN concentrates on input-output (asspecified later in the document) relations which can be represented by amuch smaller number of variables and equations than a physical basedcomputer model, optimization for the management solution will besignificantly less computationally complicated and expensive andnumerical round-off errors associated with the response coefficientmethod and huge constraint sets are avoided.

[0022] The NN/Optimizer can be interfaced directly with dataacquisition/database systems. Automatic data collection and databasesystems are already used in water resources management for storing data.NNs are perfectly suited for using this historical data for makingaccurate real-time predictions and management decisions.

SENSITIVITY

[0023] Sensitivity analysis is also a useful component for effectivewater resources management. It is often the goal of the modeler anddecision-makers to identify strong cause and effect relationships in thesystem. For example, decision-makers may want to assess the relativeimpacts of pumping and climate on water levels in the aquifer. However,non-uniqueness of the solutions in physical-based models makes itdifficult to evaluate the sensitivity of different factors on themodeled system. For example, reducing transmissivity values across themodel domain can compensate for decreasing areal recharge rates fromprecipitation.

[0024] Because the NN does not explicitly model the physics of flow, thesensitivity analysis of non-aquifer variables is simplified because thehydrogeologic parameters inherent to the physical-based model areeliminated. By systematically eliminating or disabling input variablesin the NN, the relative effect of each variable on the various outputvariables can be quantified and assessed. This provides a betterunderstanding of the natural system, which may facilitate design of amore effective hydrologic monitoring and data collection system. Thesensitivity analysis results can also improve development andcalibration of physical-based models by identifying importantinput-output calibration pairs (e.g. areal recharge rates on waterlevels in an unconfined aquifer).

[0025] The NN equations can also be combined with interpolation (e.g.kriging) and physical-based equations for predicting important physicalvariables and dynamics. Thus, the accurate predictions made with the NNcan be used as data for interpolation or physical-based models,expanding and improving prediction capability.

[0026] In summary, the NN will in many instances achieve considerablymore accurate predictive results with less developmental effort and costthan traditional physical based models. They are powerful tools forconducting sensitivity analyses that provide insight into the physicalprocesses of the groundwater/surface water system, and for identifyingimportant cause and effect relationships. They can be embedded as amodule in optimization procedures for computing superior managementsolutions for scarce and vulnerable water resources. They are alsoideally suited for automated data collection and management systems.

SUMMARY OF THE INVENTION

[0027] The present invention provides a new and useful method andapparatus for overcoming the previously discussed types of problems. Thepresent invention provides a neural network designed to do any of thefollowing

[0028] (a) predicting important groundwater/surface water-relatedoutput/state variables,

[0029] (b) optimizing groundwater/surface water-related controlvariables, and/or

[0030] (c) providing sensitivity analysis to identify physicalrelationships between input and output variables for thegroundwater/surface water system or to analyze the performanceparameters of the neural network.

[0031] In addition, the preferred version(s) of the present inventioncan provide the following additional benefits:

[0032] (a) The sensitivity analysis of the neural network systemprovides a better understanding of the natural system and can facilitatethe design of a more effective and cost efficient data monitoring andcollection system for hydrologic data. The sensitivity analysis resultsmay also be used to improve a numerical model by identifying theinterrelations between variables (e.g. precipitation and water levels).

[0033] (b) The neural network uses fewer variables and more easilymeasured or simulated data (e.g. temperature and precipitation assurrogates for evapotranspiration and recharge rates) withoutsacrificing predictive accuracy.

[0034] (c) Highly spatially variable (heterogeneous) and directionallydependent (anisotropic) physical properties, such as river bed thicknessand hydraulic conductivity, which are difficult to quantify, are notrequired by the neural network for accurately predicting output/statevariables. Highly spatially and time variable properties (e.g.soil-moisture deficit and evapotranspiration) that are difficult toquantify are also not required by the neural network for accuratelypredicting output/state variables

[0035] (d) Accurate characterization and quantification of boundaryconditions are not required by the NN for accurately predictingoutput/state variables.

[0036] (e) The neural network system is well suited for using real-timeon-line data to initialize the state-transition equations increasingpredictive accuracy.

[0037] (f) The neural network system serves as a data-driven model forautomated data collection systems (e.g. SCADA systems) that collect andstore large amounts of hydrologic data.

[0038] (g) The neural network derived transition equations can be usedto conduct formal optimization in real time using on-line automated datacollection systems. The system is able to react adaptively to new dataand information and can update optimal solutions on-line.

[0039] (h) The neural network can be used to manage a joint surfacewater/groundwater system that is interconnected by water transfer.

[0040] (i) The neural network is an excellent tool for identifying dataerrors.

[0041] Further features of the present invention will become furtherapparent from the following detailed description and the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0042]FIG. 1 is a schematic illustration of one type of neural networkarchitecture that can be used in a method and/or apparatus according tothe present invention;

[0043]FIG. 2 is a schematic illustration of an apparatus and itsoperation for prediction of output/state variables for agroundwater/surface water system, according to the principles of thepresent invention;

[0044]FIG. 3 is a schematic illustration of an apparatus and itsoperation for an optimization procedure for a groundwater/surface watersystem, according to the principles of the present invention; and

[0045]FIG. 4 is a schematic illustration of an apparatus and itsoperation for providing sensitivity analysis of input and outputvariables designed for use with a groundwater, surface water system,according to the principles of the present invention.

DETAILED DESCRIPTION

[0046] As described above, the present invention provides a new anduseful method and apparatus, based on the use of a neural network, for(a) predicting important groundwater/surface water output/statevariables, (b) optimizing groundwater/surface water control variables,and/or (c) sensitivity analysis to identify variable interdependence andphysical relationships between input and output. The principles of thepresent invention are described below in connection with a neuralnetwork designed to predict output/state variables and/or optimizingsystem control in a groundwater/surface water system. However, from thedescription, the manner in which the principles of the present inventioncan be used for various functions in a groundwater/surface water systemwill be apparent to those in the art.

[0047] The invention uses neural network technology for the difficultproblem of modeling, predicting, and managing hydrologic output/statevariables (e.g. stream flow rates, surface water levels, head orgroundwater levels, hydraulic gradients, groundwater velocities,drawdowns and water levels in pumping wells, water quality data, waterdemand, etc.) in groundwater/surface water systems. All computedvariables referenced in this document, will be called output variables.The output variables, which reflect the groundwater/surface water systemand its properties, are referred to as state variables (e.g. groundwaterelevation is a state variable, water demand based on climate conditionsis an output but not a state variable). Input and output data in thesystem comprises hydrologic, meteorological, control and water qualitydata. Hydrologic data comprises transmissivity, hydraulic conductivity,storativity, specific yield, porosity, effective porosity, aquiferthickness, leakance, leakance factor, stream flow rates, stream andriverbed thickness, slope of riverbed, surface water levels, head andgroundwater levels, hydraulic gradient, drawdowns in pumping wells,natural areal recharge, groundwater and surface water velocity,discharge, and soil moisture. Meteorological data comprisesprecipitation, wind speed, temperature, humidity, barometric pressure,and dew point. Control data comprises pumping and injection rates ofwells, surface water releases, discharge rates into natural (e.g.springs) or man-made (e.g. spreading basin) features, and extractionrates from natural or man-made features. Water quality data comprisesnatural and synthetic chemical constituents, organic constituents (e.g.bacteria, viruses), temperature, pH, electrical conductivity, and gascontent. Using inputs that may comprise state (e.g. initial groundwaterlevel, surface water level, stream discharge, water quality conditions),decision or control (e.g. pumping data, controlled releases of surfacewater, etc.), and random variables (e.g. precipitation and temperature)collected from the groundwater/surface water system with automatic datacollection systems (e.g. SCADA or Supervisory Control and DataAcquisition) or other means (e.g. manual), a neural network is trainedto predict the output variables of interest.

[0048] The equations derived from the developed neural network can beembedded into or coupled with an optimization program to identify theoptimal management solution for the resource via control variables (e.g.pumping wells). The neural network with (or without) the optimizationsystem can be coupled to an automated control/data collection/databasesystem for real-time modeling, prediction, and management ofgroundwater/surface water system In addition, the neural network can beused to conduct a sensitivity analysis to quantify and assess the effectof the various input variables, including human actions, on the outputvariables.

[0049] Initially, it is believed useful to describe certain underlyingprinciples of a neural network, its construction and usage, and todescribe the manner in which prediction of output/state variables and/oroptimization system control can be important in a groundwater/surfacewater system. It will also be apparent to those in the art that theprinciples of the present invention can be implemented on standard PCequipment.

[0050] A neural network is described by its architecture and learningalgorithm. The architecture is described by the fundamental componentsof processing elements (PEs), connection weights and layers and the wayeach component interacts with the others. A connection strategy refersto the way layers and PEs within layers are connected. A feed-forwardstrategy means that layers are connected to other layers in onedirection, from the input to the output layer. A feedback strategy meansthat some or all of the layers have changeable connections that go backto a previous layer (e.g. from the first hidden layer to the inputlayer). A fully connected strategy means that every PE in a layer isconnected to every PE in another layer.

[0051] The learning strategy refers to how the network is trained, (e.g.the functional relations of the interconnecting weights are calibrated).In supervised learning you must provide input/output pairs. The outputpatterns that you provide are compared to the output that the networkcomputes and any difference between the two must be accounted for bychanging parameters in the network. In the simplest implementation of anetwork, the delta rule is used to update the parameters (e.g. weights):

parameter_(new)=parameter_(old)+2ηεx,   (1)

[0052] The threshold function is typically represented by the sigmoidfunction, $\begin{matrix}{{{f_{j}\left( {Sum}_{j} \right)} = \frac{1}{1 + ^{- {Sum}_{j}}}},} & (4)\end{matrix}$

[0053] the hyperbolic tangent function, $\begin{matrix}{{{f_{j}\left( {Sum}_{j} \right)} = \frac{^{{Sum}_{j}} - ^{- {Sum}_{j}}}{^{{Sum}_{j}} + ^{- {Sum}_{j}}}},} & (5)\end{matrix}$

[0054] or in special cases by linear output functions.

[0055] For each pair of consecutive layers equations (2) through (5) areused.

[0056] The connection weights in the net start with random values soinitially the calculated output will not match the desired output givenin the training file. The connection weights are iteratively updated inproportion to the error.

[0057] The proportional error estimate for the output layer PE k fortraining pattern p is given as the difference between the desired outputd_(pk) from the training set and the NN calculated output o_(pk)multiplied by the derivative of the threshold function

δ_(pk)=(d _(pk) −o _(pk))ƒ_(k)′(Sum _(pk)).   (6)

[0058] The proportional error estimate for the hidden layer is given as$\begin{matrix}{\delta_{pj} = {{f_{j}^{\prime}\left( {Sum}_{pj} \right)}{\sum\limits_{k}{\delta_{p\quad k}{w_{kj}.}}}}} & (7)\end{matrix}$

[0059] Once we know the errors we use a learning rule to change theweights in proportion to the error with a step size A. The delta rule inequation (1) is applied to the output layer PEs for each input pattern,and the new connections weights between the hidden and output layerstake on the values of

w _(kj) ^(new) =w _(kj) ^(old)+ηδ_(pk) act _(pj),   (8)

[0060] and the connections weights between the input and hidden layersbecome

w _(ji) ^(new) =w _(ji) ^(old)+ηδ_(pj) x _(pi),   (9)

[0061] The threshold function is typically represented by the sigmoidfunction, $\begin{matrix}{{{f_{j}\left( {Sum}_{j} \right)} = \frac{1}{1 + ^{- {Sum}_{j}}}},} & (4)\end{matrix}$

[0062] the hyperbolic tangent function, $\begin{matrix}{{{f_{j}\left( {Sum}_{j} \right)} = \frac{^{{Sum}_{j}} - ^{- {Sum}_{j}}}{^{{Sum}_{j}} + ^{- {Sum}_{j}}}},} & (5)\end{matrix}$

[0063] or in special cases by linear output functions.

[0064] For each pair of consecutive layers equations (2) through (5) areused.

[0065] The connection weights in the net start with random values soinitially the calculated output will not match the desired output givenin the training file. The connection weights are iteratively updated inproportion to the error.

[0066] The proportional error estimate for the output layer PE k fortraining pattern p is given as the difference between the desired outputδ_(pk) from the training set and the NN calculated output δ_(pk)multiplied by the derivative of the threshold function

δ_(pk)=(d _(pk) −o _(pk))ƒ_(k)′(Sum _(pk)).   (6)

[0067] The proportional error estimate for the hidden layer is given as$\begin{matrix}{\delta_{pj} = {{f_{j}^{\prime}\left( {Sum}_{pj} \right)}{\sum\limits_{k}{\delta_{p\quad k}{w_{kj}.}}}}} & (7)\end{matrix}$

[0068] Once we know the errors we use a learning rule to change theweights in proportion to the error with a step size A. The delta rule inequation (1) is applied to the output layer PEs for each input pattern,and the new connections weights between the hidden and output layerstake on the values of

w _(kj) ^(new) =w _(kj) ^(old) +ηδ _(pk) act _(pj),   (8)

[0069] and the connections weights between the input and hidden layersbecome

w _(ji) ^(new) =w _(ji) ^(old) +ηδ _(pj) x _(pi).   (9)

[0070] After the weights are changed we re-apply the inputs and repeatthe whole process. If new data become available training can be repeatedand the NN weights can be adaptively and continuously updated. Trainingstops at the user's discretion, when the error on the test data beginsto increase, or when the training error reaches a desired minimum errorlevel.

[0071] There are several alternative architectures and learningalgorithms that can be used within a neural network system. Alternativelearning algorithms employed comprise directed random search, extendeddelta-bar-delta, conjugate gradient, Levenberg-Marquardt, generalizedNewton's method, and generalized regression method. Alternativearchitectures used comprise self-organizing maps, radial basisfunctions, modular networks, recurrent and time delay neural networks,and probabilistic neural networks.

[0072]FIG. 2 illustrates certain basic principles of a system or methodof the present invention, for predicting output/state variables in agroundwater/surface water system.

[0073] In Block 1 data collection is shown. Data can be obtained bymanual collection, simulation or on-line real-time automated datacollection system (e.g. SCADA).

[0074] Block 2 shows the pre-processing step that includes organizing,sorting, windowing, offsetting, filtering, and transforming data.

[0075] Windowing of training data is a means for increasing the amountof available data. This requires use of stress periods of variablelength, which in itself (i.e. stress period length) would serve as aninput to the NN. A stress period is defined in this document as timeduring which control and random variables remain constant. For example,assume groundwater levels were collected in all monitoring wells on thedates January 1, January 7, January 14, and January 21 of the same year,with pumping data and climate data available over this period. Then,measured water levels collected January 1 constitute the “initial”groundwater levels and are separately paired with data for January 7,January 14, and January 21 to form three distinct training records (i.e.January 1-January 7, January 1-January 4, January 1-January 21). Thus,all three training records share the same initial water level values.Each record, however, has different NN input values corresponding to thepumping rates and climate conditions for these records (e.g. obviously a3 week period will have more total precipitation than a one week period)and the stress period length (i.e. 7 days, 14 days, 21 days). The valuesof the output variables are also different for the three trainingrecords because the outputs represent the groundwater levels observed atthe end of the stress period of record.

[0076] In cases where water levels are collected manually, and thusrelatively infrequently, data windowing may be more important than incases where data is collected automatically and at higher frequency.

[0077] For cases where data is collected frequently and continuously,for example hourly, offsetting can generate large data sets. Consecutive24-hour periods would simply be offset by one hour, particularly whenhourly climate data exists. In this manner, each day does not representa calendar day, but a discrete and unique 24-hour time period. Inaddition, this method could be combined with windowing for variablelength stress periods.

[0078] Select input variables may also be transformed for NNsimplification (fewer input variables) while accounting for lag-timeeffects of input variables on final output variables. For example, forpredicting water level changes over 7 day stress periods, the dailypumping rates and precipitation values can be weighted in time, withproportionately higher weight given to values on days closer to the endof the stress period. The sum of the seven daily weighted valuesrepresents the input value for the variable. In equation form, thetransformed time weighted pumping rate may be expressed as$\begin{matrix}{P^{\prime} = {\sum\limits_{t = 1}^{t = L}{P_{t}/\left\lbrack {\left( {L + 1} \right) - t} \right\rbrack}}} & (10)\end{matrix}$

[0079] where P_(t) is the mean daily pumping rate on day t in the stressperiod, and L is the total length of the stress period.

[0080] Note from FIG. 2 (Block 2, Block 3) that preprocessing the dataand NN development/refinement is an iterative feedback process. Forexample, the NN can serve as an excellent screener for data errors. Fora large public supply wellfield case, during network validation, the NNproduced water level predictions with high errors. These outliers werefound to correspond with data records that contained transcriptionerrors.

[0081] As shown in Block 3, the neural network is designed and trainedas previously discussed. The results of the training process are thetransition equations that are then validated by additional data notincluded in the training process.

[0082] During refinement, different data sets and combinations of inputand output variables can be tried to identify a NN design that achieveslowest predictive errors for the output variables of interest. Forexample, it may be found that developing a separate NN for eachoutput/state variable may produce lowest prediction errors. Thus, theinput variables could remain constant for all NNs, with each NN havingjust one output/state variable, such as the water level at a singlelocation.

[0083] Note the general flow process showing that as new data becomeavailable, the NN is redesigned (new inputs/outputs) and/or retrained.This can be performed at select time intervals and (e.g. monthly) and/orwhen new extreme system conditions occur (e.g. 100-year storm). Thisupdating process ensures NN robustness.

[0084] In Block 4 we show the initialization process of some timedependent input/output variables. For the first stress periodprediction, these variables are initialized to real-time conditions. Forpredictions made over subsequent multiple consecutive stress periods,the initial condition for each time window is taken as the end conditionof the previous time window, which is computed by the trained NN. Thisprocess is illustrated by the feedback process from Block 5 to Block 4.For example, the real-time groundwater levels in the aquifer and climateconditions can be used to initialize the NN. The NN can then be used tomake a single stress period prediction or multiple consecutive stressperiod predictions (i.e. feedback) over a management period of interest.

[0085] Block 5 shows prediction of output/state variables as computed bythe NN equations. Separate and distinct NNs can be used to predict finaloutput/state variables. The NN equations can also be collectivelyembedded into or linked to an optimization program to identify optimalcontrol solutions. This is particularly relevant in cases wherepredicting new output/state variables is important, such as water levelsmeasured in a new observation well. Water level data would need to becollected from the new observation well. By using separate networks foreach well, however, the previously collected data in conjunction withnew data can be used to train networks to predict output water levelstates for previously existing observation wells. For the newobservation well, obviously only the new data can be used since this isthe only data for which water level measurements of the new well werecollected. Still, the “old” observation well data can still be used forNNs corresponding to the previously existing observation wells.

[0086] In Block 6 we show the option of using the NN equations topredict output/state variables between sampling points withinterpolation equations (e.g. head or water levels). Also, other outputstates (e.g. groundwater velocity) that are not explicitly computed bythe NN can be predicted by processing the NN predictions (and othernecessary data) through physical based equations. Note that thesepredictions can also be performed over multiple consecutive stressperiods, as shown by the feedback from Block 6 to Block 4.

[0087] For example, the NN can be coupled with interpolation equationsto delineate detailed two and three-dimensional head (energy) fields.That is, water levels can be estimated across a study area byinterpolating NN predicted water levels. The dynamics of the groundwatersystem can then be further defined by coupling the NN/interpolatedderived head field with a physical-based equation(s) to enable a moredetailed analysis. For example, following delineation of the head field,groundwater velocities can be computed at points of interest withDarcy's Law by inputting the delineated hydraulic gradient with theassumed hydraulic conductivity values. In the same manner, particletracking or water budgets calculations could also be performed.

[0088] If a new control variable is added to the system, such as apumping well, new data would have to be collected to include the pumpingrates of the new and existing pumping wells, and the corresponding waterlevel responses measured at locations of interest. This is necessarybecause the new well induces new stresses and corresponding systemresponses not previously observed and measured. However, the old datathat proceeds the new pumping well may still be used in many (if notmost) groundwater situations. One would simply add a new pumping rateinput variable to the neural networks. This variable would represent thepumping rate of the new well. For all records collected before the newpumping well existed, a zero pumping rate would be inputted into theneural network as the input value for this well. For all data collectedsubsequent to installation and operation of the new well, the actualrecorded pumping rates for this well would be inputted. Again, thisallows preservation of the “old” data collected previous to installationof the new pumping well(s).

[0089] As with the water level inputs, it may be that certain pumpingwells do not affect water levels at certain observation well locations,and the pumping wells could be excluded from the corresponding neuralnetworks. An obvious example of this is a multilayered unconfined andconfined groundwater system, where pumping of one aquifer does notmeasurably affect water levels in the other. Again, separate anddistinct NNs containing different control inputs will not affect formaloptimization.

[0090] For demonstrating some of the features claimed and discussedabove, two test cases are presented. In the first, a NN was trained topredict transient water levels at 12 monitoring well locations screenedat various depths in response to changing pumping and climate conditionsin a complicated multi-layered sand and limestone aquifer system. Thetrained NN was validated with ten sequential 7-day periods, and theresults were compared against both measured and numerically simulatedwater levels. The absolute mean error between the NN predicted and themeasured water levels is 0.16 meters, compared to the 0.85 metersabsolute mean error achieved with an extensively developed andcalibrated numerical flow model over the same time period. Moreimportantly, unlike the numerical model, the NN accurately mimicked thedynamic water level responses to pumping and recharge in the complicatedmulti-layered groundwater system.

[0091] As a further check for robustness, the NN's ability to accuratelypredict water level changes over multiple consecutive stress periodswithout using measured water levels for state re-initialization wasassessed. That is, measured water levels were used only forinitialization of the NN at the start of the 71-day validation period(from Feb. 4, 1998 to Apr. 15, 1998). All initial water level (state)inputs used by the NN for subsequent stress periods were predictionsmade for the corresponding previous stress period. Minimal degradationin predictive results from the continuously real-time initialized NNoccurred. The mean absolute error increasing marginally from 0.16 to0.18 meters. These results demonstrate that a NN can be used foraccurately predicting the evolution of water level changes over extendedplanning horizons using multiple consecutive stress periods. That thepredictive accuracy of the NN exhibited relatively little degradationwith time over the 2.5-month validation period suggests that theplanning horizon could be extended without significant loss in accuracy.

[0092] Similarly impressive results were obtained for predicting waterlevels in eleven large capacity public supply wells (>1,000 gpm) at theend of 24 hour pumping periods. Hourly stress periods were used, andSCADA measured water level values were used only at time 0 to initializethe NN, and the NN generated water levels were then used to reinitializethe water levels for each subsequent time step. This sequentialprocessing of the NN generated water levels was performed over a 24-hourperiod. Even though water level changes ranged over 40 feet or more ineach of the eleven wells, the NN accurately predicted the water levelchanges and achieved an average prediction error of less than 1 percent.Equally important, the percent errors did not increase with time; infact, in many cases, after a relatively large error (2%), they woulddrop significantly and remain stable through the 24^(th) hour. Theimplication of this error stability is that accurate water levelpredictions can be carried out over relatively longer time periods withno or minimal expected loss in predictive accuracy. A benefit is thatlonger-termed aquifer impacts from pumping and climate conditions canalso be predicted and assessed with confidence.

[0093] When predicting states over different prediction or planninghorizons, different stress period lengths should be tried to achievebest possible predictive accuracy. Specifically, different NNs can bedeveloped and trained for different stress period lengths (e.g. hourly,daily, weekly, etc.), and their performances compared over differentprediction or planning horizons. For example, weekly stress periods mayperform better than daily ones for making monthly predictions. Analternative is to train the NN with variable length stress periods, withthe length of the stress period used as an input to the NN.

[0094]FIG. 3 illustrates certain basic principles of a system or methodof the present invention, for optimization.

[0095] The NN equations can be embedded into any optimization modelregardless if one or more objective functions are optimized. Theseconstraints guarantee that any solution is physically feasible andreasonable.

[0096] In the system and method of FIG. 3, the Blocks 1-3 are the sameas in FIG. 2. In FIG. 2 prediction and implementation with interpolationand/or physical based equations feedback to Block 1, and in FIG. 3 theimplementation of the optimal control variables also feeds back to Block1.

[0097] Block 4 shows construction of the mathematical model foroptimization. This model comprises the physical constraints, the NNtransition equations, as well as the objective functions. In the case ofmultiple objectives the mathematical model also comprises the solutionconcept that will be discussed later.

[0098] Block 5 shows the embedding of the neural network transitionequations into the optimization model. As depicted, the differencebetween FIGS. 2 and 3 is the optimization component. As FIG. 3 shows,the NN equations in conjunction with interpolation and/or physical-basedequations, as shown in block 5, can be simultaneously embedded into anoptimization program to represent system-behavior of interest. Theseequations would represent the state-transition equations for thereal-world system. The optimization management problem is formulatedwith an objective function and management constraint set. The objectivefunction, such as minimizing pumping cost is expressed as a function ofthe control variables. Typical management constraints include minimumand maximum groundwater and surface water levels. The constraints are afunction of some combination of state, decision, and random variables.Solution of the optimization problem would identify the optimal valuesfor the control variables for maximizing or minimizing the desiredobjective function without violating any of the imposed constraints.

[0099] Block 6 is similar to Block 4 of FIG. 2. In FIG. 2 there is afeedback from Block 5 to Block 4, and in FIG. 3 a similar feedback isfrom Block 7 to Block 6.

[0100] Block 7 illustrates the optimization algorithm that is problemdependent.

[0101] After the NN weights are determined by training, the transitionequations are embedded into the optimization model as constraints. Ifall transition equations are linear then all constraints are alsolinear, otherwise they are nonlinear. The optimization managementformulation can then be solved with a variety of methods to identify theoptimal values of the decision variables that either maximizes orminimizes the objective function without violating the imposedconstraints. In a linear optimization problem, the optimal solution(assuming feasibility) is the global optimum. In non-linearoptimization, convergence to the global optimum is not guaranteed, andthe expected outcome is the so-called local optimum. In both cases, ifthe neural network's transition equations more accurately predict thenatural system's behavior of interest than the numerical model, then theoptimal solution identified with the NN equations should on average becloser to the real-world optimum than the solution identified using thenumerical model. The optimal solution can then be implemented inreal-time. The optimization can also be conducted over multipleconsecutive stress periods, with reinitialization of subsequent stressperiod conditions performed within the optimization solver. This isdepicted by the feedback loop from Block 7 to Block 6.

[0102] If there is only one criterion to be optimized (such as pumpingrate or cost) then any single-objective optimization method can be usedto solve the resulting single-objective optimization problem. If thereare multiple objectives (such as pumping rate and water level gradientas the measure of health risk), then the following procedure is used.First the Pareto-set is determined which represents those decisionalternatives that cannot be improved simultaneously in all objectives.It can be obtained by optimizing a linear combination of the objectiveswith varying weights. After the Pareto set is determined, then a filecontaining its points will replace all constraints and the point withminimal distance from the ideal point, which has the best values of theobjectives in its elements, is determined. If (p₁*,p₂*) is the idealpoint and (p₁,p₂) is any point of the Pareto-set, then with any α≧1 andweights c₁,c₂>0 we can use the distance $\begin{matrix}{\left\{ {{c_{1}{{p_{1}^{*} - p_{1}}}^{\alpha}} + {c_{2}{{p_{2}^{*} - p_{2}}}^{\alpha}}} \right\}^{\frac{1}{\alpha}}.} & (11)\end{matrix}$

[0103] Alternatively instead of minimizing distance from the ideal pointconflict resolution methods can be used. Let (p₁*,p₂*) be the pointhaving the worst possible objective values in its elements. Then we canmaximize the distance (11) from this point or maximize the Nash product:

|p ₁ −p _(1*)|^(c) ^(₁) |p ₂ −p ₂*|^(c) ^(₂) ,   (12)

[0104] or maximize the improvement from this worst outcome point in agiven direction, where the direction depends on the relative importanceof the objectives. As an alternative method we might find the point thathas equal weighted distances from the best values of the differentobjectives, or we can find the point such that the linear segmentconnecting it with the worst outcome point divides the feasible set intotwo parts of equal area. In the multiobjective optimization literaturemethod (11) is known as the distance based method with theMinkowski-distance. Method (12) is called the general Nash bargainingsolution, and the three concepts mentioned afterwards are known as theKalai-Smorodinsky, the equal sacrifice, and the area monotonicsolutions.

[0105] Block 8 shows the implementation of the results. The actualoptimization procedure, which is problem dependent, provides the optimalvalues of the decision variables. The corresponding output/statevariable values as well as all relevant numerical consequences have tobe computed. In addition directions are given to the decision makers tothe practical implementation of the optimal solutions.

[0106]FIG. 4 illustrates certain basic principles of a system or methodof the present invention, for sensitivity analysis.

[0107] Blocks 1 through, 4 are the same as discussed for FIG. 2 withidentical interrelations.

[0108] Block 5 shows the procedure when an output variable is calculatedby using all input variables as well as without a particular variable.The comparison of the results shows the sensitivity of the outputvariable with respect to the selected input. Systematically, selectedinput nodes are eliminated or disabled in the NN and the NN output/statevariables are recomputed. A comparison in predictive accuracy achievedwith the complete NN and the incomplete NN is then performed. From this,the relative importance of each input variable on each output variablecan be assessed and quantified. Note the feedback loop to collectingreal-world data and repeating the sensitivity analysis. As the systemchanges over time, the sensitivity of output/state to different inputsmay also change. For example, a recently dry river may increase theeffect of a pumping well on the aquifer at a location near the dryriver.

[0109] The utility of using a NN for conducting a sensitivity analysisfor better understanding the physical relationships between input andoutput/state variables in a hydrologic system was demonstrated for themultilayered hydraulically connected sand and limestone aquifer systemdiscussed earlier in paragraph 0097. Groundwater levels are monitored inboth aquifers with monitoring wells, and public supply wells arescreened in the deeper leaky aquifer. A NN was trained to predict thefinal groundwater levels in both aquifers (output) as a function of theinputs, consisting of pumping rates, initial groundwater levels, andprecipitation. The sensitivity analysis determined that pumping mostaffected groundwater levels in the deeper aquifer. Conversely, thesensitivity analysis demonstrated that water levels in the shallowaquifer are most affected by precipitation rather than pumping.Additionally, the analysis was used to determine the degree of influencethat each pumping well has on groundwater levels at each location. Thiscould be particularly critical in hard rock systems, whereinterconnections between different fractures are important forunderstanding the hydrogeologic framework.

[0110] Although the examples given here pertain to predictinggroundwater output/states, the predictive accuracy of the NN has alsobeen demonstrated for surface water systems. For example, the waterlevels in a series of springs in Florida and streamflows in New Jerseywere accurately predicted using the NN approach. In addition,time-varying chloride concentrations associated with saltwater upconinginduced by pumping of a coastal aquifer in Massachusetts were accuratelypredicted with a NN.

[0111] In light of the foregoing, it is believed the importantcontributions that result from a method and apparatus constructed, orpracticed, according to the principles of the present invention, can bereadily appreciated.

[0112] Accordingly, the foregoing detailed description provides a methodand apparatus, based on the use of a neural network, and one exemplaryexample of such a method and apparatus, for (a) predicting importantgroundwater/surface water related output/state variables, (b) optimizinggroundwater/surface water related control variables, and/or (c)sensitivity analysis to identify physical relationships between inputand output/state variables for the groundwater/surface water system orto analyze the performance parameters of the neural network. With theforegoing disclosure in mind, the manner in which the principles of thepresent invention can be applied to the construction and/or practice ofvarious types of groundwater/surface water systems, will be apparent tothose in the art.

1. A method of predicting output/state variables in agroundwater/surface water system, comprising the steps of (a) providingand training a neural network with hydrologic and control data for agroundwater/surface water system, and (b) operating the trained neuralnetwork to predict output/state variables in the groundwater/surfacewater system, the output variables comprising one or more of thefollowing: surface water elevations, surface water flow rates,groundwater heads and elevations, groundwater gradients, groundwatervelocities, and chemical concentrations.
 2. A method as defined in claim1, wherein the step of providing and training the neural network furthercomprises providing and training the neural network with meteorologicaldata.
 3. A method as defined in claim 2, wherein the step of providingand training the neural network further comprises providing and trainingthe neural network with water quality data.
 4. A method as defined inclaim 1, wherein the step of providing and training the neural networkfurther comprises providing and training the neural network with waterquality data.
 5. A method as defined in claim 1, wherein thegroundwater/surface water system is a groundwater system, and whereinthe neural network is operated to predict output/state variables, atleast one of which comprises groundwater elevation/head.
 6. A method asdefined in claim 1, wherein the groundwater/surface water system is asurface water system, and wherein the neural network is operated topredict output/state variables, at least one of which comprises surfacewater elevations.
 7. Apparatus for predicting output/state variables ina groundwater/surface water system, comprising (a) a neural network thathas been provided and trained with hydrologic and control data for agroundwater/surface water system, and (b) the trained neural networkbeing configured to predict output/state variables in thegroundwater/surface water system, the output variables comprising one ormore of the following: surface water elevations, surface water flowrates, groundwater head and elevations, groundwater gradients,groundwater velocities, and chemical concentrations.
 8. A method ofproviding sensitivity analysis for a neural network for agroundwater/surface water system, comprising the steps of (a) providingand training a neural network with input data for a groundwater/surfacewater system, the trained neural network configured to produce outputcomprising output/state variables of the groundwater/surface watersystem, and (b) operating the neural network to define relationshipsbetween selected input data and selected output variables.
 9. A methodas set forth in claim 8, wherein the input data comprises a plurality ofinput variables of the groundwater/surface water system, and the step ofoperating the neural network comprises operating the neural network todefine physical relationships between a least one of the real inputvariables and at least one of the output variables of thegroundwater/surface water system.
 10. A method of providing transitionequations for an optimization procedure for a groundwater/surface watersystem, comprising the steps of (a) providing a trained neural networkthat has been trained with hydrologic and pumping data for thegroundwater/surface water system and (b) configuring the neural networkto produce transition equations that can be used either as functionroutines or constraints in the optimization procedure.
 11. An apparatusfor use in an optimization procedure for a groundwater/surface watersystem, comprising a trained neural network that has been trained withhydrologic and control data for the groundwater/surface water system andis configured to produce transition equations that can be used either asfunction routines or constraints in the optimization procedure.